1. Field
This disclosure relates to a system for performing home health care monitoring. More specifically, this disclosure relates to a system for performing home health care monitoring using depth-enhanced video content analysis.
2. Background
Many people, particularly the elderly, need to take certain medications and maintain a certain level of physical activity to stay healthy. In addition, as people get older, there tends to be a greater likelihood of accidents or sudden emergencies, such as falling down, stroke, heart attacks, emergencies due to a failure to take medicine, etc. In addition, many elderly people suffer from depression, dementia, or other conditions that alter their daily behavior. Preventing and treating these accidents, emergencies, and conditions is very important but can be very expensive. For example, people may move into assisted living quarters or nursing homes, or may hire hospice care or home help.
To assist elderly patients, various less expensive alternatives have been used. For example, using Lifeline® by Philips, elderly patients can simply push a button if an emergency situation occurs, which alerts emergency response personnel. Other systems may use video or other electrical equipment to monitor a patient's health. For example, U.S. Pat. No. 5,544,649 to David et al., published on Aug. 13, 1996, describes a patient health monitoring system that includes video cameras in a person's home that connect to a central station for health care workers to monitor. Additional medical condition sensing and monitoring equipment, such as blood pressure, pulse, and temperature monitoring devices, may also be used at the patient's home. However, such a system uses a fairly large number of health care workers per patient, and therefore can still be fairly expensive and requires constant attention of the workers.
In hospital environments, cameras that perform automatic detection of certain patient behaviors have been proposed. For example, U.S. Patent Application Publication No. 2012/0075464 to Derenne et al., published on Mar. 29, 2012, proposes the use of video cameras in the hospital environment to determine certain patient conditions, such as whether a patient is sleeping, exiting a bed, walking, or falling. Proposed cameras for such a system include RGB cameras with depth sensors that may be used to provide full-body 3D motion detection, among other things.
However, systems such as proposed by Derenne et al., while monitoring individual events, do not monitor collective household activity such as to be expected of an elderly patient living at home. In addition, systems that provide full 3D motion detection, for example by analyzing three-dimensional data for all parts of a video scene (e.g., all pixels of a series of video frames) can be computationally complex, and may require special software and processing capability beyond the scope of traditional two-dimensional monitoring schemes, which may further increase the expense of such monitoring.
An example of a two-dimensional video content analysis (VCA) system is described in U.S. Pat. No. 7,932,923, issued to Lipton et al. on Apr. 26, 2011 (the '923 patent), the contents of which are incorporated herein by reference in their entirety. Some existing systems use RGB (red green blue), CMYK (cyan magenta yellow key), YCbCr, or other sensors that sense images in a two-dimensional manner and perform analysis of those images to perform object and event detection. Other existing systems use depth sensors, to generate three-dimensional data or depth maps, which are then analyzed using different software in order to perform object and event detection. In some ways, the systems that use depth sensors are more accurate than the two-dimensional systems. For example, the depth sensor systems may obtain more accurate three-dimensional information, and may deal better with occlusions. However, depth data and images determined by depth sensor systems are generally lower in resolution than RGB data, and may therefore include fewer details than RGB images. In addition, depth sensors are a relatively new technology for video analysis, and are still prone to error in determining three-dimensional coordinates. Further, certain information resulting from depth sensors often remains incomplete, such as depth information for objects with specularities, or depth information for featureless surfaces extracted from stereo.
Certain systems may combine both depth and RGB data in order to perform analysis on complex three-dimensional scenes. For example, as described in U.S. Pat. No. 7,831,087, depth data and optional non-depth data are used to generate a plan-view image, which plan view image can then be analyzed by classifying objects in the plan view image. However, systems such as this, which perform complex analysis on depth data and optional additional data in order to perform object detection or event detection, still suffer from the problems above relating the drawbacks of depth sensor systems. For example, some of the depth data may be missing or may be inaccurate, resulting in an analysis of faulty data. In addition, performing analysis on three-dimensional data generally requires more complex algorithms and may require a complete re-design of hardware and/or software that performs the analysis, compared to more traditional two-dimensional image analysis systems.
The embodiments described here address some of these problems of existing systems, and provide a new and simplified way to use depth data to assist in image analysis and video content analysis throughout a patient home environment. As a result, better home health care monitoring can be achieved using automated systems that are more accurate and reliable than prior systems.